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PREFACE 


The  current  study  explores  the  operational  utility  of  fusing  synthetic  aperture  radar 
(SAR)  imagery  and  digital  terrain  map  (DTM)  data.  Specifically,  the  two-dimensional 
display  of  SAR  imagery  was  compared  against  the  2  V2-dimensional  display  of  SAR 
overlaid  on  corresponding  DTM  data. 

The  study  was  conducted  with  the  joint  participation  of  the  Israel  Air  Force  (LAF) 
Human  Factors  Engineering  Branch  (HFEB)  and  the  Crew  Systems  Interface  Division  of 
the  AFRL’s  Human  Effectiveness  Directorate  (AFRL/HEC).  The  research  took  place  in 
Israel  during  the  period  of  19  August  through  17  September  1998.  The  USAF 
participation  was  performed  under  Work  Unit  71841044,  “Advanced  Crew  Systems  for 
Reconnaissance,  Surveillance  and  Target  Acquisition.”  Mr.  Gilbert  G.  Kuperman 
(AFRL/HECA)  and  Lt  Col  Itzhak  Nadler  (Chief,  IAF  HFEB)  served  both  as  principal 
investigators  and  as  the  technical  project  officers  for  the  DEA  for  their  respective 
countries.  This  study  was  carried  out  with  the  contractual  support  of  Synergy  Integration, 
Ltd.,  Tel  Aviv,  Israel,  in  the  area  of  image  processing,  and  by  PAMAM  Human  Factors 
Engineering,  Ltd.,  Ramat  HaSharon,  Israel,  in  the  areas  of  test  instrument  design,  data 
collection,  analysis,  and  documentation. 

In  any  exploration  of  this  type,  the  true  knowledge  resides  with  the  subject  matter 
experts  of  the  operational  units.  The  authors  are  greatly  indebted  to  the  young  men  and 
women  of  the  Israeli  Ground  Corps  Command  and  the  Israel  Air  Force,  who  shared  their 
expertise  so  willingly  with  the  experimenters  in  support  of  this  research. 

Individual  thanks  are  due  to  Ms.  Ayelet  Oettinger  (PAMAM),  who  so 
professionally  supported  the  data  collection  and  analysis  portions  of  the  study  and  to  both 
Mr.  Jacob  Silbiger  and  Mr.  Victor  Ben-Ezra  (Synergy),  who  prepared  the  imagery  and 
terrain  data  for  use  as  stimuli. 

Lastly,  special  thanks  must  be  expressed  to  Lt  Col  Danni  Zimra,  Israeli  Ministry 
of  Defense,  Directorate  of  Defense  Research  and  Development  (MAFAT),  for  his 
assistance  in  obtaining  the  SAR  imagery  employed  in  this  study. 
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INTRODUCTION 


Background 

Synthetic  aperture  radar  (S  AR)  sensors  offer  two  compelling  advantages  over 
conventional  (electro-optical)  sensing  technologies:  stand-off  range  and  adverse  weather 
capabilities.  SAR  images  can  be  formed  with  effectively  no  loss  in  resolution  out  to  the 
limits  of  the  system’s  stabilization  and  motion  compensation  capabilities.  SAR  sensors 
can  “see”  through  clouds  and  through  light  rain.  Further,  depending  on  their  coverage 
mode  and  data  processing  limitations,  SAR  sensors  can  be  capable  of  high  area  coverage 
rates.  These  attributes  make  SAR  imaging  a  valuable  resource  for  tactical  and  theater 
airborne  reconnaissance,  surveillance  and  target  acquisition  applications. 

The  air  forces  of  both  the  United  States  and  of  the  State  of  Israel  have  great 
interest  in  exploiting  these  capabilities.  The  USAF  has  operational  SAR  capabilities  in 
the  B-1B,  F-15E,  J-STARS,  and  U-2  systems  and  plans  to  include  SAR  as  a  primary 
imaging  mode  in  both  the  Global  Hawk  and  Dark  Star  families  of  uninhabited  air 
vehicles.  The  IAF  has  operational  SAR  capabilities  in  their  Phantom  2000  and  F-15I 
multi-role  aircraft  and  has  other  SAR  capabilities  in  development. 

SAR,  however,  is  a  non-literal  imaging  sensor.  That  is,  the  imagery  produced  by  a 
SAR  does  not  resemble  a  photograph  taken  of  the  same  scene.  The  intensity  values  in  the 
SAR  image  are  proportional  to  the  radar  cross  sections  of  the  corresponding  points  in  the 
ground  scene  (and  not  to  their  visible  wavelength  reflectance).  The  impulse  response 
function  of  the  SAR  (the  fundamental  determinant  of  system  resolution)  includes  side 
lobes.  Thus,  the  return  from  a  point  on  the  ground  may  include  energy  contributed  by 
adjacent  scatterers.  The  “shadows”  in  a  SAR  image  are  caused  by  the  active  illumination 
of  the  scene  by  the  emitting  radar  (and  not  by  the  sun  angle).  The  perspective  of  a  SAR 
image  is  that  of  an  observer  looking  down  onto  the  scene  from  directly  above,  while  it  is 
being  illuminated  by  the  radar  from  one  side  (the  location  of  the  SAR). 

Because  of  the  non-literal  nature  of  the  SAR  image,  operational  questions  exist 
regarding  how  well  an  imagery  analyst  (IA)  can  orient  it  against  a  map  reference.  A 
fundamental  imagery  exploitation  task  is  to  confirm  (or  plot)  the  actual  ground  coverage 
of  a  collected  image  against  a  map  reference.  Several  other  standard  imagery  exploitation 
tasks  (e.g.,  landform  analysis,  traversability  studies)  require  that  the  operator  interpret  the 
image  so  as  to  assess  the  basic  geologic  and  terrain  characteristics,  including  judgments 
of  the  heights  of  terrain  features  and  the  grades  of  slopes.  Further,  orientation  may 
require  the  IA  to  locate  salient  terrain  features  and  to  match  them  against  their  map 
references.  Understanding  of  the  terrain  contributes  significantly  to  the  establishment  and 
maintenance  of  situational  awareness,  affording  the  context  within  which  other  imagery 
interpretations  may  be  made.  The  human  operator  is  unique  in  having  the  ability  to  apply 
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contextual  information  to  the  interpretation  of  complex  visual  stimuli  (such  as 
reconnaissance  imagery). 

S  AR  is  not  the  only  technology  which  may  support  these  operational 
requirements.  Digital  terrain  map  (DTM)  data,  consisting  of  elevation  “posts,”  equally 
spaced  in  latitude  and  longitude,  provide  another  source  of  information  regarding  the 
heights  and  slopes  of  the  terrain.  DTM  data  can  be  viewed  in  two  dimensions  (2-D)  as 
elevation  contours  or  as  a  continuous  tone  depiction,  in  which  elevation  is  coded  by 
luminance  values  or  colors.  Two-D  image  displays  may  be  rotated  so  that  north  (or  any 
arbitrary  direction)  is  toward  the  top  of  the  display.  Alternatively,  DTM  data  may  be 
displayed  in  2  V2-D,  in  which  a  3-D  “model”  of  the  terrain,  with  a  shading  scheme 
applied  as  if  it  were  illuminated  by  the  sun,  is  projected  onto  the  2-D  display  surface. 

Two  and  one-half-D  DTM  displays  may  be  rotated  in  both  azimuth  and  elevation  to 
change  the  effective  viewpoint  of  the  observer. 

Fusion  also  offers  potential  capabilities  to  support  enhanced  orientation, 
situational  awareness,  and  information  extraction  capabilities.  Disparate  data  sources, 
such  as  SAR  imagery  and  DTM  elevations,  may  be  combined  (overlaid)  so  as  to  support  a 
2  V2-D  display  of  the  SAR  images. 


Objective  and  Approach 

The  objective  of  this  study  was  to  perform  an  operational  assessment  of  the 
relative  utility  of  2-D  and  2  */2-D  displays  of  SAR  imagery.  In  the  2-D  case,  the  SAR 
images  were  viewed  conventionally.  In  the  2  V2-D  case,  the  SAR  image  was  overlaid  on 
the  corresponding  DTM  model.  Subject  matter  experts  (SMEs),  military  IAs  assigned  to 
the  Israel  Air  Force  (IAF),  the  Intelligence  Command,  or  the  Ground  Corps  Command, 
performed  orientation  and  information  extraction  tasks  using  both  display  formats.  The 
study  was  conducted  at  the  facilities  of  Synergy  Integration,  Ltd.,  Tel  Aviv,  the  contractor 
to  the  IAF’s  Human  Factors  Engineering  Branch  (IAF  HFEB),  with  the  support  of 
PAMAM  Human  Factors  Engineering,  Ltd.,  the  contractor  supporting  the  Air  Force 
Research  Laboratory  (AFRL),  during  the  period  19  August  through  17  September  1998. 
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METHOD 


Imagery 

The  S  AR  imagery  used  in  this  experiment  was  acquired  by  a  developmental 
sensor  flown  on  the  Israel  Aircraft  Industry’s  Boeing  737  multi-mode  radar  testbed 
aircraft.  The  imagery  had  a  nominal  resolution  of  1.2  m.  The  imagery,  in  detected  form, 
had  a  nominal  dynamic  range  of  8  bits  (or  256  gray  levels).  All  imagery  was  acquired  at 
high  grazing  angles  (approximately  45  degrees). 

Three  imagery  swaths  were  provided  by  the  Israeli  Ministry  of  Defense.  The  first 
included  coverage  of  the  Armored  Command  Museum  at  Latrun.  The  second  included 
the  area  of  Rosh  Ha’ayin  and  the  third  included  coverage  of  Ben  Gurion  International 
Airport.  Thirty-eight  stimulus  images  were  extracted  from  the  Latrun  and  Rosh  Ha’ayin 
swaths.  Six  images,  used  only  for  familiarization  with  the  task  and  practice  with  the 
apparatus,  were  extracted  from  the  Ben  Gurion  coverage. 

The  Rosh  Ha’ayin  and  the  Latrun  swaths  differed  in  scale.  In  the  Rosh  Ha’ayin 
swath,  each  centimeter  of  the  displayed  image  represented  approximately  60  meters  on 
the  ground.  In  the  Latrun  swath,  each  centimeter  of  the  image  represented  approximately 
92  meters  on  the  ground.  As  a  result,  the  width  of  Rosh  Ha’ayin  swath  was 
approximately  1  km  by  1  km  and  that  of  the  Latrun  swath,  approximately  1.5  km  by  1.5 
km.  The  size  of  each  of  the  images  was  700  by  700  pixels. 

Selection  of  “Targets” 

The  experimental  design  was  constrained,  to  some  extent,  by  the  coverage  of  the 
available  imagery.  Since  the  objective  of  the  experiment  was  to  investigate  the  effect,  on 
both  orientation  and  situational  awareness,  of  S  AR  imagery  overlaid  on  a  three- 
dimensional  terrain  elevation  database  and  viewed  in  a  2  V2-D  display,  no  buildings  were 
included  as  targets.  A  senior  and  highly  experienced  IAF IA  first  determined  the 
coverage  of  the  SAR  imagery  against  a  1:50,000  scale  survey  map.  Features  (such  as 
river  bends,  confluences/divergences  of  streams,  the  intersections  of  dirt  roads,  etc.)  were 
selected  from  the  map  information  for  use  as  designation  “targets,”  and  their  Universal 
Transverse  Mercator  (UTM)  coordinates  were  read  and  recorded.  These  same  features 
were  then  located  within  the  SAR  images,  and  the  corresponding  pixel  location  was  read 
and  recorded.  This  process  was  repeated  until  all  38  stimulus  targets  and  the  six  practice 
targets  had  been  selected.  The  target  location  coordinates  were  maintained  as  the  “school 
solution”  for  scoring  the  accuracy  of  the  designation  portion  of  the  task.  The  imagery  was 
then  divided  into  22  matched  pairs  (one  half  of  each  pair  to  be  presented  in  2  V2-D  and 
the  other  half  in  2D).  The  pairings  were  made  on  the  basis  of  containing  similar  targets 
within  similar  backgrounds. 
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Overlay  of  SAR  Imagery  onto  DTM  Data 


Commercial  off-the-shelf  software  (MultiGen  E  Pro,  from  MultiGen,  Inc.,  San 
Jose,  California)  was  used  by  Synergy  Integration,  Ltd.,  Tel  Aviv,  to  convert  the  SAR 
pixel  coordinates  into  UTM  coordinates — the  reference  system  used  for  the  DTM  data. 
Multiple  control  points  were  selected  from  each  of  the  SAR  images  and  their  geographic 
reference  locations  were  carefully  determined  from  the  map.  A  transformation  program 
using  these  control  points  was  used  to  convert  every  pixel  location  into  its  corresponding 
UTM  coordinates.  One  SAR  image  from  each  matched  pairing  was  then  overlaid  onto 
the  corresponding  DTM  elevation  data  (using  the  same  software  package).  The  product 
of  this  procedure  was  a  2  V2-. D  representation  of  the  area  (as  compared  to  the  2-D 
representation  of  the  original  SAR  imagery). 

No  additional  exaggeration  to  the  elevation  data  was  introduced.  Thus,  the 
displayed  image  of  the  overlaid  SAR  and  DTM  depicted  ground  distances  (z  and  y)  and 
heights  (z)  in  the  ratios  of  1:1:1. 

Experimental  Apparatus 

All  data  collection  was  performed  at  the  facilities  of  Synergy  Integration,  Ltd. 

The  images  were  displayed  and  designation  coordinates  and  response  times  were  recorded 
using  a  Silicon  Graphics,  Inc.  (SGI)  ONYX  graphics  workstation  equipped  with  an 
Infinite  Reality  Engine  multi-processor.  The  workstation  was  also  equipped  with  a  SGI 
model  CM2187ME  533  mm  (21  inch)  diagonal  color  monitor.  The  display  resolution 
(full  screen)  was  1280  by  1024  pixels.  The  brightness  and  contrast  controls  of  the  display 
were  set  by  the  Experimenter.  The  apparatus  was  located  in  a  laboratory  setting  and  was 
used  to  support  both  stimulus  preparation  and  data  collection.  All  stimulus  imagery  was 
displayed  using  commercial  off-the-shelf  software  (the  VEGA  general  visualization 
environment  from  Paradigm  Simulations,  Inc.,  Dallas,  Texas).  The  displayed  image  (700 
by  700  pixels)  was  approximately  200  by  200  mm  (8  by  8  inches)  on  the  monitor. 

Subject  Matter  Experts 


Five  enlisted  LAs  from  the  Israel  Defense  Force  Ground  Corps  Command’s 
Imagery  Analysis  Unit,  three  LAs  from  the  IAF,  and  two  weapon  systems  officers  (WSOs) 
of  the  IAF  served  as  subject  matter  experts  (SMEs).  All  were  male.  They  ranged  in  age 
from  19  to  25  years.  Their  experience  in  tactical  imagery  exploitation  ranged  between  6 
months  and  6  Vi  years.  Four  of  the  IAs  and  both  WSOs  had  some  experience  with  SAR 
imagery;  all  of  them  had  experience  in  the  exploitation  of  electro-optical  (photography 
and  television)  sensor  collections,  and  all  had  previous  experience  in  performing  softcopy 
imagery  exploitation.  None  of  the  SMEs  had  previous  experience  in  exploiting  high 
resolution  SAR  imagery  (as  was  used  in  the  present  study). 
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All  SMEs  had  6/6  (20/20)  vision,  uncorrected  or  corrected,  and  all  had  received 
formal  military  training  in  imagery  analysis  during  a  three-month  duration  service  school. 


The  SME  Task 

Figure  1  depicts  the  sequence  of  events  which  composed  the  experimental  task. 
Upon  arrival  at  the  laboratory  facility,  the  SMEs  were  informed  as  to  the  purpose  of  the 
study  and  instructed  regarding  the  conduct  of  the  experiment.  The  instructions  to  the 
SMEs  explicitly  placed  primary  emphasis  on  the  accurate  performance  of  the  designation 
component  of  the  task  but  also  emphasized  the  requirement  to  complete  the  task  as 
rapidly  as  possible.  The  instructions  also  included  the  caution  that  the  imagery  was  more 
recent  than  the  map  and  might  contain  (extensive)  differences  with  respect  to  the  addition 
of  man-made  structures  such  as  buildings  and  roads. 

Information  regarding  the  SME’s  background,  training  and  imagery  exploitation 
experience  was  elicited  through  a  brief  questionnaire  which  included  questions  regarding 
their  formal  training  in  the  exploitation  of  SAR  imagery  and  their  experience  in 
interpreting  softcopy  and  SAR  imagery.  The  SME  was  then  seated  at  the  graphics 
workstation. 

At  the  beginning  of  the  task,  each  SME  was  shown  an  extract  from  a  1:50,000 
scale,  color,  topographic  survey  map  of  Israel.  The  map,  oriented  north-up  and  covering 
approximately  2  km  by  2  km  in  area,  had  been  annotated  to  depict  the  coverage  of  a  SAR 
image  at  a  different  orientation  and  included  a  red  dot  marking  the  location  of  a  target. 
This  map  allowed  the  SME  to  understand  the  relative  differences  in  coverage  between 
succeeding  map  extracts  and  their  corresponding  SAR  images.  They  were  then  instructed 
in  the  use  of  the  apparatus  for  the  imagery  orientation  and  target  designation  portions  of 
the  task.  The  practice  images  were  used  to  allow  the  SMEs  to  gain  proficiency  in  the  use 
of  the  equipment,  the  orientation  and  target  designation  components  of  the  experimental 
task,  and  the  nature  of  the  SA  questions.  Any  remaining  questions  that  the  SMEs  might 
have  had  regarding  the  task  were  answered  by  the  Experimenter  at  this  time.  When  the 
SMEs  reported  that  they  were  confident  in  the  execution  of  the  task,  the  data  collection 
trials  were  initiated. 

At  the  beginning  of  each  of  the  38  data  collection  trials,  a  1:1  scale  extract  from  a 
1:50,000  scale,  color,  topographic  survey  map  of  Israel  was  provided  to  the  SME.  The 
map  extract  was  always  oriented  north-up  and  covered  approximately  2  km  by  2  km  in 
area.  The  header  on  the  map  copy  described  the  type  of  target  to  be  located  (e.g.,  dome, 
intersection  of  a  dirt  road  and  a  stream,  etc.),  while  the  exact  location  of  the  specific 
target  of  interest  was  depicted  on  the  map  itself  by  a  small  red  dot.  The  map  extracts 
were  mounted  as  successive  pages  in  a  flip  chart-type  booklet.  The  Experimenter 
initiated  each  trial  by  depressing  a  specific  function  key  on  the  keyboard.  The  SME  was 
permitted  15  seconds  for  map  study.  During  this  interval,  the  image  display  region  was 
blank  (showing  a  solid,  medium  luminance,  light  blue  field).  The  Experimenter  informed 
the  SME  whether  the  current  trial  was  a  2-D  or  2  14-D  display  format.  The  SAR  image 
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containing  the  target  then  appeared  on  the  workstation  display.  The  images  were  always 
presented  so  that  the  radar  shadows  pointed  toward  the  bottom  of  the  display  (i.e.,  as  if 
the  radar  were  illuminating  the  ground  from  along  the  top  edge  of  the  display).  No 
restriction  was  placed  on  the  viewing  distance  between  the  SME  and  the  workstation 
monitor. 


Next  Target  and  Image 


Figure  1:  Flow  diagram  of  the  SME’s  task. 

The  SMEs  were  permitted  up  to  3  minutes  (180  seconds)  during  which  they  were 
required  to  orient  themselves  to  the  S  AR  image  in  the  context  provided  by  the  map 
information  (which  was  available  throughout  the  trial),  to  locate  the  pre-briefed  target, 
and  to  designate  the  target.  At  the  completion  of  the  tasks,  the  display  automatically  went 
blank  and  performance  time  was  recorded.  If  the  SME  did  not  respond  within  180 
seconds,  the  display  went  blank  and  the  trial  was  recorded  as  having  “timed  out.”  During 
this  3-minute  period  the  SMEs  could  use  the  left  and  right  arrow  keys  on  the  workstation 
keyboard  to  rotate  the  image  through  a  full  360  degrees  of  azimuth.  The  up  and  down 
arrow  keys  “tipped”  the  image  through  90  degrees  of  “elevation.”  Rotation  in  both 
azimuth  and  elevation  were  continuous  and  could  be  applied  in  any  combination. 

For  each  SME,  half  the  stimulus  images  were  presented  in  overlay  on  the  DTM 
elevation  data.  In  these  cases,  rotation  of  the  displayed  image  produced  a  2  V2-D  view. 
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In  the  other  half  of  the  trials,  a  2-D  view  was  presented.  The  arrow  keys  could  still  be 
used  for  tip  and  rotation,  but  no  elevation  data  were  overlaid  on  the  SAR  images.  The 
mouse  was  used  to  drive  an  “arrow”  cursor  on  the  display  to  any  point  on  the  image. 
When  the  SME  had  located  the  target,  the  ENTER  key  on  the  keyboard  was  used  to 
record  the  target  location  into  the  data  file  for  that  trial.  (The  use  of  the  ENTER  key  was 
preferred  to  the  use  of  a  mouse  key  in  order  to  prevent  involuntary  motion  of  the  mouse 
cursor  during  designation.) 

Upon  designation,  the  display  was  blanked,  and  the  location  of  the  designated 
point  was  automatically  recorded,  along  with  the  time  duration  between  stimulus  onset 
and  the  act  of  target  designation.  The  SME  then  flipped  the  page  in  the  map  booklet  (thus 
precluding  any  further  reference  to  the  map)  and  found  two  questions  regarding  the  image 
presented  during  the  just-completed  trial.  These  SA  questions  dealt  either  with  absolute 
or  relative  terrain  height  judgments  or  with  the  relative  location  of  other  objects  in  the 
SAR  image.  The  answers  to  the  questions  were  recorded  manually  by  the  Experimenter. 
This  allowed  for  immediate  answers  to  any  SME  requests  for  clarification  of  the  S  A 
questions. 


Situational  Awareness  Questions 

Two  situational  awareness  (SA)  questions  were  developed  by  the  Experimenters 
for  each  target  image.  These  questions  dealt  with  absolute  or  relative  terrain  height 
judgments  (e.g.,  which  bank  of  a  stream  was  higher;  which  slope  of  a  dome  was 
steepest?)  with  the  direction  of  objects  (e.g.,  what  was  the  direction  of  the  stream?)  or 
with  the  relative  location  of  objects  in  the  SAR  image  (e.g.,  in  which  direction  from  the 
stream  bend  were  two  large  buildings?).  Each  of  the  SA  questions  was  presented  in 
multiple  choice  form.  Three  possible  answers  to  each  question  were  presented,  and  the 
SME  had  to  select  the  correct  one.  No  time  limit  was  imposed  in  answering  these 
questions. 

Once  the  S  A  questions  had  been  answered,  the  trial  was  completed.  The  SME 
then  indicated  readiness  to  proceed  with  the  next  trial.  This  sequence  was  repeated  until 
all  38  images  had  been  presented  to  the  SME.  The  SMEs  were  given  a  short  break  after 
each  group  of  8  to  12  trials  (while  the  Experimenter  loaded  a  different  SAR  swath). 

Rating  Scale  Questions 

After  all  38  stimulus  images  had  been  presented,  the  SME  was  asked  to  complete 
a  series  of  rating  scale  questions  regarding  overall  impressions  of  the  task  and  of  the  two 
different  display  formats.  Each  scale  consisted  of  seven  points  with  semantic  anchors  at 
each  endpoint  (as  shown  in  Figure  2). 
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Figure  2:  Rating  scale  with  semantic  anchors. 

A  rating  of  “1”  always  meant  that  the  2  V2-D  display  greatly  degraded  the  SME’s 
ability  to  perform  the  referenced  function,  while  a  rating  of  “7”  always  meant  that  the 
2  l/2-D  display  greatly  enhanced  that  ability.  The  first  group  of  questions  dealt  with 
comparisons  between  the  2-D  and  2  V2-D  display  formats  with  respect  to  performing 
general  orientation,  assessing  the  structure  of  the  terrain,  assessing  differences  in  terrain 
heights,  and  assessing  terrain  slopes.  The  next  scale  required  the  SME  to  rate  the  utility 
of  the  2  V2-D  display  format  in  supporting  general  imagery  interpretation  tasks.  Another 
set  of  questions  related  to  S  A.  The  SME  was  asked  about  the  differences  between  the 
2-D  and  2  V2-D  display  formats  in  supporting  giving  answers  to  the  SA  questions.  The 
SMEs  were  also  asked  to  comment  on  whether  they  relied  primarily  on  the  map  extract  or 
on  the  S  AR  imagery  in  answering  these  questions.  They  were  also  requested  to  comment 
on  the  relevance  of  the  S  A  questions  to  their  current  military  duties.  Provision  was  also 
made  for  the  SMEs  to  record  any  overall  impressions  or  comments  regarding  the  entire 
experiment.  The  completed  questionnaires  for  all  SMEs  are  presented  in  Appendix  A. 

Upon  completion  of  the  rating  scales,  data  collection  was  ended,  and  the  SME  was 
thanked  for  participation  in  the  experiment.  Each  SME  participated  for  approximately  2 
hours,  including  instruction,  practice,  data  collection,  and  completion  of  the 
questionnaire. 


Experimental  Design 

A  mixed,  within-subject  experimental  design  was  employed.  Half  of  the  SMEs 
were  presented  with  one  half  of  the  matched  SAR  image  pairs  overlaid  onto  the  DTM 
data;  the  other  half  of  the  SMEs  were  presented  with  the  alternate  half  of  the  image  pairs 
presented  in  overlay.  Half  of  the  SMEs  were  presented  with  the  experimental  imagery  in 
the  reverse  order  from  that  presented  to  the  other  SMEs.  This  counterbalance  was  to 
protect  against  learning  effects.  Thus,  there  were  four  unique  combinations  of  imagery 
presentation:  order  of  presentation  and  DTM  or  non-DTM  underlay  (the  independent 
variable  of  interest). 
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ANALYSIS  AND  RESULTS 


Accuracy  Computation 

The  distance  between  the  actual  location  of  each  target  and  the  point  designated 
by  each  subject  during  each  trial  was  calculated  as  follows.  Targets  were  divided  into 
two  categories,  targets  with  an  accurate  location  (e.g.,  road  junction,  stream  intersection) 
and  targets  whose  exact  geolocation  was  less  certain  (e.g.,  dome). 

For  each  target  category,  a  different  scoring  system  was  used. 

1 .  For  targets  with  an  accurate  location: 

•  A  designation  distance  of  less  than  0.8  cm  on  the  screen  was  considered 
accurate.  This  translates  to  a  distance  of  less  than  48  m  in  the  Rosh 
Ha’ayin  swath,  and  of  less  than  73.6  m  in  the  Latrun  swath. 

•  A  designation  distance  of  0.8- 1 .6  cm  was  considered  a  partially  accurate 
designation.  This  translates  to  a  distance  of  48-96  m  in  the  Rosh  Ha’ayin 
swath,  and  of  73.6-147.2  m  in  the  Latrun  swath. 

•  A  designation  distance  larger  than  1.6  cm  was  considered  an  error.  This 
translates  to  a  distance  of  more  than  96  m  in  Rosh  Ha’ayin  and  more  than 
147.2  m  in  Latrun. 

2.  For  targets  of  less  certain  location: 

•  A  designation  distance  of  less  than  1.6  cm  on  the  screen  was  considered 
accurate  (less  than  96  m  in  the  Rosh  Ha’ayin  swath  and  less  than  147.2  m 
in  the  Latrun  swath). 

•  A  designation  distance  of  1. 6-4.8  cm  was  considered  a  partially  accurate 
designation  (96-288  m  in  Rosh  Ha’ayin  and  147.2-441.6  m  in  Latrun). 

•  A  designation  distance  larger  than  4.8  cm  was  considered  an  error  (more 
than  288  m  in  Rosh  Ha’ayin  and  more  than  441.6  m  in  Latrun). 

A  score  of  2  was  assigned  to  each  accurate  designation,  a  score  of  1  to  each 
partially  accurate  designation,  and  a  score  of  0  to  errors. 

The  differences  between  ground  distances  which  represent  a  certain  level  of 
accuracy  in  the  Latrun  and  the  Rosh  Ha’ayin  swaths  are  due  to  differences  in  scale 
between  the  two.  It  was  decided  to  base  error  computation  on  designation  distances  in  the 
image  rather  than  on  ground  distances,  because  they  provide  a  better  representation  of 
subjective  accuracy. 
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Target  Designation  Accuracy  Results 


Individual  Scores 

Accuracy  scores  for  each  SME  and  each  image  are  presented  in  Appendix  B. 

Overall  Accuracy  Scores 

The  mean  accuracy  score  for  the  2-D  display  was  1.33  and  for  the  2  Vz-D  display, 
1.39.  This  difference  is  not  statistically  significant. 

Analysis  of  the  accuracy  scores  by  swaths  (Latrun  or  Rosh  Ha’ayin)  did  not  yield 
any  significant  main  effect  or  interaction. 

Selected  Targets  Accuracy  Scores 

It  was  hypothesized  (based  on  subjects’  comments — see  Appendix  A)  that  while 
overlaid  S  AR-DTM  images  may  not  be  particularly  advantageous  in  flat  areas  or  in  areas 
that  contain  salient  human-made  features,  they  may  provide  an  “edge”  in  areas  which  are 
mountainous  and  without  salient  features.  To  test  this  hypothesis,  a  subset  of  images  was 
selected  which  contains  mountainous  areas,  without  salient  human-made  features.  The 
selected  targets  included  12  images  from  the  Latrun  swath  and  8  images  from  the  Rosh 
Ha’ayin  swath. 

For  the  selected  images,  the  average  accuracy  scores  were  1.13  for  2-D  and  1.20 
for  the  2  Vi-D.  This  difference  is  not  statistically  significant. 

Analysis  of  the  accuracy  scores  of  the  selected  images  by  swaths  (Latrun  or  Rosh 
Ha’ayin)  did  not  yield  any  significant  main  effect  or  interaction. 

It  may  be  seen  that  the  scores  for  the  selected  targets  are  lower  than  the  scores  for 
the  whole  set,  indicating  that  these  sections  tended  to  be  more  difficult  than  the  average. 

Difficulty  Levels 

In  order  to  rule  out  possible  floor  and  ceiling  effects,  accuracy  scores  were 
computed  without  the  easiest  and  without  the  most  difficult  targets. 

The  targets  were  divided  into  three  difficulty  levels,  according  to  their  average 
distance  score. 

•  Difficult:  0-0.8  (6  targets). 

•  Medium:  0.9- 1.8  (25  targets). 

•  Easy:  1.9-2  (7  targets). 
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The  six  “difficult”  and  the  seven  “easy”  targets  were  eliminated  and  only  the 
medium  difficulty  level  targets  were  analyzed.  The  difference  between  the  2-D  and 
2  V2-D  average  scores  was  somewhat  larger  than  for  the  whole  set  (1.33  and  1.46, 
accordingly).  However,  this  difference  failed  to  reach  statistical  significance. 


Learning  Effects 

Based  on  subjects’  comments  and  on  the  fact  that  2  V2-D  displays  were  new  to  all 
subjects,  it  was  hypothesized  that  2  V2-D  image  responses  may  have  been  affected  by 
learning  during  the  experiment  more  than  those  for  the  2-D  images. 

Figure  3  represents  average  performance  for  each  of  the  38  trials.  (Note  that 
because  of  the  counterbalancing  procedures,  the  accuracy  score  of  each  trial  is  based  on 
more  than  one  image.)  It  can  be  seen  that,  while  the  first  2  V2-D  trials  tend  to  be  less 
accurate  than  the  first  2-D  trials,  the  last  trials  tend  to  be  more  accurate.  The  interaction 
between  trial  and  accuracy  did  not  reach  statistical  significance. 


DTM  — ■—  NO  DTM - Linear  (DTM)  -  -  -  Linear  (NO  DT~M)~] 


Figure  3:  Accuracy  scores  by  order  of  trials. 


Individual  Differences 

The  mean  performance  of  some  of  the  SMEs  seems  to  have  benefited  from  the 
overlay  of  the  SAR-DTM,  while  the  mean  performance  of  others  was  degraded.  An 
overall  picture  of  individual  performance  can  be  seen  in  Appendix  B.  Table  1  presents 
general  IA  experience,  SAR  experience,  accuracy  scores  with  2-D  and  2  V2-D  images, 
and  the  difference  scores.  The  general  trend  seems  to  be  that  SMEs  with  little  or  no  SAR 
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experience  benefited  more  from  the  SAR-DTM  overlay  than  SMEs  with  SAR  experience. 
It  may  also  be  noted  that,  on  the  average,  the  two  WSOs  performed  more  accurately  than 
all  IAs  and  that  the  best  IA  was  also  the  most  experienced  one. 


Table  1.  Experience  Levels  and  Accuracy  Scores  of  the  SMEs 


SME 

General 

experience 

(years) 

SAR 

experience 

(years) 

Accuracy 
score  2D 

Accuracy 
score  2  Vi-D 

Difference 

4  -  IA 

0.5 

0 

0.89 

1.00 

0.11 

3-  IA 

1.5 

0 

1.00 

1.37 

0.37 

2-IA 

2.5 

0 

0.95 

1.39 

0.44 

5  -  IA 

2.5 

0 

1.21 

1.42 

0.21 

1  -IA 

1 

1 

1.00 

1.32 

0.32 

8  -  LA 

1.5 

1 

1.47 

1.26 

-0.21 

9  -  IA 

2 

1.5 

1.42 

1.37 

-0.05 

6-WSO 

WSO 

2 

2.00 

1.89 

-0.11 

10-WSO 

WSO 

3.5 

1.63 

1.53 

-0.11 

7  -  IA 

6.5 

6.5 

1.74 

1.37 

-0.37 

Target  Designation — Speed 

The  time  of  target  designation  was  recorded  for  each  target.  If  the  subject  failed  to 
respond  after  180  seconds,  the  display  automatically  went  blank  and  a  “timeout”  was 
recorded.  Only  13  timeouts  were  recorded  for  the  total  of  380  trials  by  the  10  SMEs. 

This  low  timeout  rate  suggests  that  the  180  seconds  allowed  for  the  orientation  and 
designation  components  of  the  imagery  exploitation  task  was  appropriate  to  the 
complexity  of  the  task. 

•  Four  out  of  the  13  timeouts  were  associated  with  a  single  image  that  was 
particularly  hard. 

•  More  than  half  of  the  timeouts  (eight)  were  produced  by  2  of  the  10  SMEs, 
perhaps  indicating  lack  of  experience  or  excessive  emphasis  on  accuracy. 

Designation  time  tended  to  be  longer  for  overlaid  SAR-DTM  images.  The  mean 
response  time  for  the  2-D  images  was  51.9  seconds  and  for  the  2  l/t-D,  60.6  seconds. 

This  difference  is  statistically  significant,  F(l)=24.71,  p=0.0008. 
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Designation  times  for  the  Latrun  swath  (51.79)  were  significantly  shorter  than  for 
the  Rosh  Ha’ayin  swath  (60.68),  F(l)=22.5,  p=0.001 1.  The  shorter  response  times  in  the 
Latrun  swath  may  be  due  to  the  higher  availability  of  salient  human-made  features  in  the 
images  of  the  Latrun  area. 


Situational  Awareness 


SA  Scores 


Each  trial  was  followed  by  two  SA  questions.  A  score  of  1  was  assigned  to  each 
correct  answer,  and  0,  to  wrong  answers.  SA  scores  were  computed  for  trials  with  correct 
and  partially  correct  target  designations  only.  The  final  SA  scores  were  computed  as  the 
sum  of  points  for  each  trial. 

Overall  SA  Scores 

The  mean  SA  score  for  the  2-D  images  was  1.08,  and  for  the  2  !/2-D  images,  1.04. 
This  difference  is  not  significant. 


Selected  SA  Questions 

Based  on  SMEs  comments  (Appendix  A),  it  was  hypothesized  that  the  overlaid 
SAR-DTM  may  affect  only  features  that  are  related  to  height  (e.g.,  slopes,  height 
differences)  and  not  other  features  (e.g.,  direction  of  roads).  Therefore,  questions 
regarding  height  differences  were  analyzed  separately.  However,  this  treatment  did  not 
produce  significant  differences  between  the  2-D  and  the  2  ‘/2-D  images. 


SA  bv  Swath 

No  significant  differences  were  found  between  the  SA  scores  of  the  Latrun  and  the 
Rosh  Ha’ayin  swaths. 


Types  of  SMEs 

The  SMEs  who  participated  in  the  study  came  from  of  two  groups:  eight  image 
analysts  (IAF  and  ground  forces)  and  two  WSOs  (IAF  aircrew).  A  statistical  analysis  was 
performed  for  all  10  SMEs,  and  another  was  performed  for  the  8  LAs  only.  This  treatment 
did  not  produce  significant  differences  between  the  2-D  and  the  2  ‘/2-D  designation 
accuracy  scores  or  SA  ratings. 
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Rating  Scales 


The  first  four  questions  on  the  rating  scale  dealt  with  the  strength  of  the  SMEs 
preference  for  either  the  2  Vz-D  or  the  2-D  S  AR  display  format  in  the  context  of 
supporting  the  IA’s  ability  to  orient  to  the  terrain  scene.  The  first  question  addressed 
general  orientation,  the  second  addressed  the  assessment  of  landforms/terrain  structure, 
the  third,  understanding  of  terrain  height  differences,  while  the  fourth  explored 
understanding  of  differences  in  terrain  slopes.  As  depicted  in  Figure  4,  the  SMEs  as  a 
group  expressed  a  marked  preference  for  the  2  V2-D  display  format.  (In  Figure  4,  a  mean 
rating  of  4.0  reflects  no  preference  between  the  two  formats.) 

The  fifth  rating  scale  required  the  SMEs  to  express  their  preference  in  the  context 
of  the  utility  of  the  display  format  to  support  imagery  interpretation  in  general.  A 
preference  for  the  2  V2-D  format  was  found. 

The  sixth  rating  scale  explored  the  two  display  formats  in  the  context  of  SA. 
Again,  a  preference  for  the  2  V2-D  was  elicited. 


MEAN  RATING 


Figure  4:  Mean  ratings  for  responses  to  six  questions. 

All  ratings  were  significantly  higher  than  the  mean  score  (4.0).  Table  2  presents 
the  statistical  summary  for  10  SMEs. 
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Table  2.  Mean  Ratings  and  T  Scores  for  the  Six  Rating  Scale  Questions 


Question 

Mean  rating 

T  score 

Probability 

1 

4.6 

2.64 

0.0331 

2 

5.4 

4.24 

0.0038 

3 

6.1 

13.74 

0.0001 

4 

5.5 

3.53 

0.0096 

5 

5.4 

5.22 

0.0012 

6 

5.0 

2.55 

0.0379 

Observations 

Before  discussing  the  implications  of  the  results  from  the  formal  measures  used  in 
the  study,  some  observations  on  the  part  of  the  Experimenters,  made  during  the  data 
collection  runs,  may  give  the  reader  insight  into  the  study. 

•  None  of  the  SMEs  had  any  apparent  difficulty  in  employing  the  display/controls 
mechanization  (arrow  keys,  mouse,  enter  key)  used  in  this  study. 

•  Although  none  of  the  SMEs  had  any  experience  in  the  exploitation  of  high 
resolution  SAR,  they  were  all  able  to  complete  the  target  designation  task  without 
any  reported  difficulty. 

•  All  IAs  had  received  training  in  landform  and  traversability  analysis  as  part  of 
their  IA  school  curriculum. 

•  Some  SMEs  indicated  that  the  effective  usage  of  2  Vz-D  images  may  require 
additional  practice,  and  perhaps,  even  formal  training. 

•  Wide-ranging,  individual  differences  were  observed  with  regard  to  the  strategies 
employed  by  the  SMEs  in  viewing  the  SAR  display.  Some  SMEs  physically 
rotated  the  paper  map  to  match  the  orientation  of  the  SAR  (regardless  of  whether 
DTM  data  were  available).  This  kept  the  radar  shadows  pointing  toward  the 
bottom  of  the  display — a  technique  that  IAs  are  taught  to  employ  to  avoid  a 
“false”  reversal  in  apparent  elevation  /  depression  of  the  scene.  Others  appeared 
to  first  rotate  the  SAR  display  (again  regardless  of  the  format)  and  then  to  quickly 
tilt  the  displayed  image,  apparently  to  gain  an  appreciation  for  terrain  relief. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Conclusions 

High  resolution  SAR  imagery,  collected  at  high  grazing  angles,  does  not  appear  to 
present  any  of  the  difficulties  conventionally  associated  with  low  and  medium  resolution 
non-literal  imagery — at  least  in  the  context  of  the  present  salient  landform  designation 
and  terrain-based  SA  tasks.  This  also  suggests  that  only  minimal  impact  to  the  training 
support  system  may  be  encountered  as  these  systems  become  operational. 

Designation  scores  with  the  overlaid  SAR-DTM  imagery  (2  l/2-D)  produced 
slightly  higher  accuracy  scores  than  with  SAR  alone  (2-D).  However,  these  differences 
were  small  and  did  not  reach  statistical  significance.  The  general  pattern  of  results  did 
not  change  when  only  selected  targets,  which  contained  mountainous  areas  and  no  salient 
human-made  features,  were  analyzed.  The  elimination  of  the  most  difficult  and  the 
easiest  trials  from  the  statistical  analysis  increased  the  differences  between  the  2-D  and 
the  2  Vz-D  scores,  but  this  difference,  too,  failed  to  reach  statistical  significance.  Several 
factors  may  have  affected  the  potential  effects  of  an  overlaid  SAR-DTM  imagery  on  the 
accuracy  of  target  recognition: 

•  The  sets  of  SAR  swaths  used  in  the  study  were  rather  limited  in  size  and  included 
only  small  areas,  which  were  both  mountainous  and  free  of  salient  human  made 
objects.  Hence,  the  number  of  sections  in  which  the  SAR-DTM  overlay  could 
provide  significant  advantages  was  rather  small  and  the  variety  was  very  limited. 

•  Because  of  the  limited  width  of  each  swath  and  the  small  variety  of  useful  sections 
the  size  of  the  area  displayed  during  each  trial  was  significantly  smaller  than  the  size 
of  area  which  IAs  use  in  their  regular  routine.  This  made  the  use  of  terrain  features 
more  difficult  than  usual. 

•  The  use  of  the  overlaid  SAR-DTM  seems  to  require  some  training.  This  was 
indicated  by  the  results,  which  show  a  larger  improvement  in  SAR-DTM 
performance  than  in  SAR  alone,  and  was  pointed  out  by  some  of  the  SMEs  as  well. 

Response  times  were  approximately  17  percent  longer  for  the  2  Vz-D  trials  than 
for  the  2-D  trials.  This  is  not  surprising,  given  that  the  2  Vz-D  images  contain  more 
information.  Additionally,  during  the  2  Vz-D  trials  subjects  made  more  extensive  use  of 
the  tilt  option,  which  provided  them  with  different  views  of  the  terrain,  whereas  tilting  the 
2-D  images  was  possible  but  did  not  provide  any  extra  information. 

Situation  awareness,  as  measured  by  the  questions  at  the  end  of  each  trial,  did  not 
benefit  from  the  overlay  of  SAR-DTM.  Two  reasons  may  have  affected  the  results.  First, 
the  answers  to  the  SA  questions  could  be  extracted  from  the  maps,  as  well  as  from  the 
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SAR  images.  At  the  end  of  the  experiment,  SMEs  were  asked  about  the  extent  to  which 
their  SA  answers  were  based  on  the  SAR  imagery  as  compared  with  the  map  (see 
Appendix  A).  Most  SMEs  reported  that  the  maps  were  an  equal  or  a  dominant  source  of 
SA  information.  Obviously,  the  use  of  the  map  obscures  SAR  imagery  effects.  Secondly, 
although  all  IAs  considered  the  SA  questions  as  relevant  to  their  jobs,  they  also  indicated 
that  the  level  of  detail  required  tended  to  be  higher  than  is  usually  required  on  the  real 
“object  recognition”  job  (e.g.,  comparing  the  slopes  of  two  adjacent  domes).  Several 
SMEs  indicated  that  this  level  of  detail  would  be  more  relevant  for  determining 
traversability.  Hence,  some  of  the  time  SA  questions  were  perceived  as  an  additional, 
secondary  task  rather  than  as  an  integral  part  of  the  main  target  acquisition  task. 

Individual  performance  differences  were  quite  large  and  seem  to  be  related  to  the 
level  of  experience.  Interestingly,  the  more  experienced  SAR  users  seem  to  have 
benefited  less  from  the  SAR-DTM  overlay  than  the  inexperienced  subjects.  However, 
these  findings  were  not  significant  and  require  further  investigation. 

In  their  subjective  ratings  at  the  end  of  the  experiment,  SMEs  expressed  their  faith 
in  the  2  Vz-D  imagery’s  potential  as  an  aid  for  image  analysis — to  improve  SA,  enhance 
general  orientation,  increase  understanding  the  structure  of  terrain  and  facilitate 
perceiving  height  and  slope  differences.  (See  Appendix  A  for  detailed  answers.) 

Recommendations 

•  Future  studies  should  include  exploration  of  the  2  Vz-D  SAR  and  other  sensor 
display  formats  to  support  IA  confidence  in  performing  S A  and  information 
extraction  tasks.  (This  recommendation  is  based  on  observation  of  the  SMEs’ 
strategies  in  carrying  out  the  tasks.) 

•  The  use  of  a  DTM  underlay  should  be  studied  in  conjunction  with  various  types  of 
sensor  imagery  under  conditions  where  sensor  imagery  may  disappear  or  fade  out 
(e.g.,  passing  through  a  cloud,  degraded  conditions  for  thermal  imagery).  It  is 
hypothesized  that  under  these  conditions,  the  DTM  may  serve  as  an  anchor, 
prevent  loss  of  orientation,  and  thus,  enhance  orientation  and  object  recognition 
performance. 

•  SME  training  and  individual  differences  have  played  an  important  role  in  the 
present  study.  These  issues  require  further  investigation. 
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APPENDIX  A 


QUESTIONNAIRES 

(Translated  from  the  Original  Hebrew) 


1.  Did  you  prefer  either  of  the  SAR  display  formats  for  use  in  performing  the  terrain 
interpretation  subtasks  of: 


a.  General  orientation  between  the  map  and  the  image? 


s— -x— -s 


STRONGLY  1  2 

PRFFF.R  1  1 

3  4  5  6 

1  1  1  1 

7  STRONGLY 
_ l  PREFER 

2D 

FORMAT 

(NEUTRAL: 

NO  PREFERENCE) 

2l/2D 

FORMAT 

(s  =  1  standard  deviation;  X 

=  mean  of  group  rating) 

b.  Understanding  the  general  structure  of  the  terrain? 

STRONGLY  1  2 

PRF.FF.R  1  1 

s - x - s 

3  4  5  6 

1  1  1  1 

7  STRONGLY 
_ 1  PREFER 

2D 

FORMAT 

(NEUTRAL; 

NO  PREFERENCE) 

21/2D 

FORMAT 

c.  Understanding  differences  in  terrain  heights? 


s - X - s 


STRONGLY  1 

2 

3  4  5 

6 

7  STRONGLY 

PREFER  I _ 

1 

-  1  1  1 

1 

1  PRFFFR 

2D 

(NEUTRAL; 

21/2  D 

FORMAT 

NO  PREFERENCE) 

FORMAT 

d.  Understanding  differences  in  terrain  slopes? 


STRONGLY  1 

2 

s - 

3  4  5 

-X - 

6 

— -S 

7  STRONGLY 

PREFER  I _ 

i 

1  1  1 

1 

i  PRFFFR 

2D 

FORMAT 

(NEUTRAL; 

NO  PREFERENCE) 

21/2D 

FORMAT 
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SME  comments  regarding  interpretation  of  terrain: 

SME  1.  - 

SME  2.  2  ‘/2-D  was  helpful  only  when  there  were  height  differences. 

SME  3.  No  difference  in  flat  areas. 

SME  4.  - 

SME  5.  - 

SME  6.  - 

SME  7.  2  V2 — D  may  help  in  areas  with  height  differences  and  without  other  salient  cues. 
SME  8.  - 

SME  9.  Orientation  is  normally  based  on  salient  features  and  not  on  terrain  structure;  my 
attempt  to  use  the  2  V2-D  did  not  succeed. 

SME  10.  One  needs  to  get  acquainted  with  the  2  */2-D  format. 


2.  Which  S  AR  display  format  would  be  of  greatest  utility  in  performing  general  imagery 
analysis  tasks? 


s - X - s 

STRONGLY 
PREFER 
21/2D 
FORMAT 

SME  comments  regarding  utility  of  the  SAR  display  formats  to  general  imagery  analysis 
tasks: 

SME  1 .  Presently  irrelevant;  may  be  of  help  for  understanding  traversability  and 
detecting  obstacles. 

SME  2.  - 

SME  3.  Useful  for  orientation. 

SME  4.  Useful  in  areas  without  man-made  cues. 

SME  5.  May  be  of  help  for  detecting  traversability  problems. 

SME  6.  (WSO)  height  and  slope  differences  are  not  very  important  for  my  work. 

SME  7.  May  improve  the  general  understanding  of  terrain  structure,  and  in  particular, 
traversability. 

SME  8.  - 

SME  9.  Get  a  better  idea  about  terrain  structure. 

SME  10.  (WSO)  May  support  orientation,  especially  where  there  are  height  differences 

3.  Were  the  tasks  used  in  this  experiment  relevant  to  your  normal  duties  as  an  imagery 
analyst? 

SME  comments  regarding  the  relevance  of  the  experimental  tasks  to  normal  duties: 

SME  1.  Not  relevant  for  SAR  interpretation,  in  which  there  is  no  strong  emphasis  on  the 
terrain. 


STRONGLY  1 
PREFER  L 
2D 

FORMAT 


(NEUTRAL; 

NO  PREFERENCE) 
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SME  2.  Yes,  for  the  process  of  initial  orientation. 

SME  3.  May  help  to  extract  target  accurate  coordinates. 

SME  4.  The  usage  of  map  and  SAR  was  relevant. 

SME  5.  The  parallel  use  of  the  map  and  the  SAR  was  relevant. 

SME  6.  As  a  WSO,  it  may  help  orientation,  but  is  more  important  for  las. 

SME  7.  Yes,  the  orientation  part. 

SME  8.  May  help  in  defining  traversability  and  obstacles. 

SME  9.  Yes,  orienting  between  a  map  and  a  SAR  image. 

SME  10.  (WSO,  not  AI)  -  may  support  the  orientation  process. 

4.  Was  there  a  difference  between  the  two  the  SAR  display  formats  for  use  in  responding 
to  the  situational  awareness  (SA)  questions  asked  after  each  trial? 

s - X - s 

STRONGLY  12  3  45  67  STRONGLY 

DISAGREE  I _ I _ I _ 1 _ | _ | _ 1  AGREE 

(NEUTRAL; 

NO  PREFERENCE) 


SME  comments  regarding  responding  to  SA  questions: 

SME  1.  DTM  is  not  necessary  when  there  are  2-D  salient  cues. 

SME  2.  - 

SME  3.  - 

SME  4.  - 

SME  5.  The  conclusions  from  the  map  were  sometimes  quantitatively  different  than 
those  drawn  from  the  SAR;  this  posed  a  decision  problem. 

SME  6.  Yes,  the  orientation  and  location  of  specific  areas  on  the  ground. 

SME  7.  2  Vi[— D]  helps  when  there  are  height  differences. 

SME  8.  Directions,  with  the  map;  for  heights,  the  SAR-DTM  has  an  absolute  advantage. 
SME  9.  - 

SME  10.  Used  the  2  V4[-D]  for  difficult  3-D  areas. 


5.  Were  your  responses  to  the  SA  questions  based  (primarily)  on  the  SAR  image  (either 
display  format)  or  on  the  paper  map? 

SME  responses  regarding  use  of  SAR  imagery  versus  paper  map  in  responding  to  SA 
questions: 

SME  1.  Both. 

SME  2.  Both. 

SME  3.  Extracted  directions  from  the  map,  altitude  and  slopes  from  the  SAR. 

SME  4.  Map 

SME  5.  Map,  except  for  questions  which  had  no  answer  on  the  map. 
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SME  6.  Map. 

SME  7.  Both,  the  SAR-DTM  provides  a  more  accurate  view  of  the  terrain. 

SME  8.  Map. 

SME  9.  Both. 

SME  10.  Primarily  the  map. 

6.  Were  the  terrain  features  (“targets”)  that  were  included  in  the  target  designation  and 
SA  tasks  relevant  to  your  duties  as  an  imagery  analyst? 

SME  comments  regarding  relevance  of  terrain  features: 

SME  1.  Sometimes  (e.g.,  intersections  of  dirt  roads). 

SME  2.  Not  really. 

SME  3.  May  be  relevant  for  terrain  interpretation  but  not  for  tactical  interpretation. 
SME  4.  - 

SME  5.  Relevant  only  for  traversability  research. 

SME  6.  Not  relevant  for  a  WSO. 

SME  7.  features  are  relevant  for  orientation. 

SME  8.  Prefers  2-D  because  he  is  used  to  it. 

SME  9.  For  softcopy  IA  the  orientation  part  will  be  secondary  because  of  automatic 
system  support. 

SME  10.  Not  really;  these  details  are  relevant  for  orientation,  but  not  for  interpretation. 


7.  General  comments: 

SME  1.  - 

SME  2.  - 

SME  3.  - 

SME  4.  - 

SME  5.  In  general,  this  looks  very  promising. 

SME  6.  - 

SME  7.  - 

SME  8.  2  V2-D  may  help  in  locating  obstacles  and  landing  areas. 

SME  9.  - 

SME  10.  The  requirement  to  rotate  the  SAR  image  north-up,  (in  order  to  prepare  for  SA 
questions)  is  an  obstacle,  because  we  usually  view  them  from  the  direction  from 
which  they  were  taken. 
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APPENDIX  B 


The  following  table  presents  the  accuracy  scores  for  all  subjects  in  all  trials. 

“Swath”:  is  either  Latrun  or  Rosh  Ha’ayin. 

“Target”:  is  the  serial  number  of  the  S  AR  and  map  section  which  was  used  for  one  trial. 

“Target  type”:  0  =  targets  with  an  accurate  location;  1  =  targets  that  were  spread  on  a 
larger  area. 

“DTM”:  0  =  SAR  only;  1  =  S  AR+DTM.  For  each  target,  the  left  side  of  the  table 

presents  the  scores  of  the  SMEs  who  performed  with  SAR  only  and  the  right 
side  presents  the  scores  of  the  SMEs  who  performed  the  same  target  with 
SAR+DTM — or  vice  versa. 

“SME”:  1  -  Israeli  Ground  Corps  Command  Unit  (1-5) 

2  -  Israel  Air  Force  Imagery  Analysis  (7  -  9) 

3  -  Weapon  systems  officers  (6,  10) 

“0”  score,  in  boldface  type,  indicates  “timeout”  (no  target  designation  response  during 
180  seconds);  13  timeouts  were  recorded  during  the  whole  study. 

“x”  -  indicates  missing  data 
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Table  B-l.  Accuracy  Scores  for  all  Subjects  in  all  Trials 


GLOSSARY 


2-D 

two-dimensional 

2  Vi-D 

2  ^-dimensional 

AFRL 

Air  Force  Research  Laboratory  (United  States) 

cm 

centimeter 

D 

dimension(al) 

DEA 

Data  Exchange  Annex 

DTM 

digital  terrain  map 

HFEB 

Human  Factors  Engineering  Branch  (of  the  IAF) 

IA 

imagery  analyst 

IAF 

Israel  Air  Force 

IOF 

International  Opportunities  Fund 

J-STARS 

Joint  Surveillance  and  Target  Acquisition  Radar  System 

km 

kilometer 

m 

meter 

mm 

millimeter 

pixel 

picture  element 

SA 

situational  awareness 

SAR 

synthetic  aperture  radar 

SME 

subject  matter  expert(s) 

UTM 

Universal  Transverse  Mercator 

WSO 

weapon  systems  officer 
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